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MidTerm  Overall 
Evaluation  Report 


PROPOSAL:  2000000141 
TITLE:  Electronic  Behavioral  Health  “Health-E” 


ACCOMPLISHMENTS 

This  project  has  progressed  ahead  of  expectations  for  most  components  and  has  successfully 
responded  to  meet  real  world  operational  needs.  Evolving  external  challenges,  elaborated  on  below, 
have  resulted  in  a  reprioritization  of  project  deliverables  and  a  focus  on  support  services  that  can  be 
provided  without  Internet  connectivity.  This  has  not  been  to  the  exclusion  of  Internet  based  health 
care  services,  rather  as  an  additional  important  focus.  Thus,  at  this  point  the  project  is  a  two 
pronged  effort.  One  major  thrust  remains  development  of  Internet  behavioral  health  services.  The 
other  major  effort  is  focused  on  meeting  behavioral  health  needs  off-line.  These  are  mutually 
supporting  foci  which  have  added  to  project  complexity. 

This  project  remains  under  the  direction  of  the  original  project  PI.  We  are  utilizing  a  broad 
spectrum  of  behavioral  health  providers  to  assist  with  content  development  and  project  deliverable 
prioritization.  The  providers  involved  in  the  various  project  components  overlap  in  their 
responsibilities  and  interests  and  include  both  a  local  group  at  Ft  Lewis  and  an  AMEDD  wide  group 
that  includes  the  senior  leadership  of  the  behavioral  health  specialties.  The  Internet  based 
development  is  focused  on  standardization  of  content,  functionality,  and  cross  platform 
compatibility  as  we  explore  the  potential  for  Army  Knowledge  Online  (AKO)  and  TricareOnline 
integration.  Specialty  representatives  from  the  Psychology,  Psychiatry,  Social  Work  and  Substance 
Abuse  programs  have  been  recruited  and  engaged  in  active  development  within  their  areas. 
Subspecialties  within  each  department  have  also  been  involved  with  efforts  directed  to  leverage  the 
areas  with  greatest  potential  for  development.  Within  the  Behavioral  Health  Clinic  this  has  included 
enabling  clinic  forms,  materials,  and  briefings  for  the  Internet  to  support  soldiers  and  commanders 
on  Ft  Lewis.  The  Wellness  program  has  modified  significant  portions  of  their  program  for 
downloading,  and  delivery  through  CDs  with  the  first  sample  CDs  having  been  produced.  The 
development  cuts  across  the  Behavioral  Health  Science  Service  Line  at  MAMC  and  into  the  Ft  Lewis 
activities  particularly  within  the  Suicide  Prevention  Program  which  we  are  supporting  through 
Internet  enabling  of  their  programs  and  briefings. 

With  a  multidisciplinary  approach,  the  research  team  has  identified  factors  critical  to  the  success  of 
a  useful,  comprehensive  electronic  behavioral  health  clinic.  From  this  we  have  prioritized  needs  and 
initiated  development  of  project  components,  elaborated  on  below,  which  include  outcome  measure 
development,  wireless  deployment  integration,  development  of  a  disaster  Response  Tool-Kit”  with 
the  VA,  efforts  to  integrate  cognitive  testing,  and  development  of  a  deployable  electronic  behavioral 
health  record.  These  efforts  have  demonstrated  their  viability  and  are  evolving,  with  the  support  of 
the  senior  behavioral  health  leadership,  to  an  integrated  enterprise  solution. 


Pi's  Accomplishment  Evaluation:  :  I 
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PROBLEMS 


During  the  tenure  of  this  project  numerous  changes  have  occurred  that  directly  and  indirectly 
impact  upon  its  delivery.  Formalization  of  the  DITSCAP  and  SSAA  requirements  has  been 
implemented.  DOD  and  other  websites  have  been  identified  as  increased  targets  of  attack  resulting 
in  multiple  new  security  processes  and  tightening  of  existing  processes.  The  9-11  events  have  had 
both  a  direct  and  indirect  influence  on  this  project.  Directly  this  and  other  terrorist  activities  have 
resulted  in  increased  network  and  Internet  downtime,  increased  limitations  on  flexibility  of 
implementation,  and  website  configuration  difficulties  as  patches  and  new  policies  are  implemented 
and  the  “bugs”  are  worked  out.  Indirectly  this  has  resulted  in  personnel  distraction  and  delays  in 
response  to  requests  for  support  to  rapidly  evolving  requirements.  These  requirements  have  taxed 
our  MAMC  web  personnel  and  challenged  their  support  abilities.  These  processes  have  impacted  the 
delivery  of  Internet  based  services  to  challenge  the  targeted  model  of  interactive  Internet  services 
from  our  local  site.  In  particular,  web  content  is  now  all  being  cached  at  Ft  Huachuca  preventing 
efficient  interactive  communication  such  as  bulletin  boards.  This  project  continues  to  be  challenged 
by  long  delays  in  order  processing  and  installation,  particularly  due  to  our  bringing  in  new 
technologies  and  requirements  that  MAMC  personnel  are  not  familiar  with.  These  external  factors 
have  not  stopped  efforts  to  provide  Internet  based  services  but  have  impacted  on  what  is  feasible  to 
deliver  within  present  constraints  and  led  to  greater  exploration  of  services  beyond  MAMC. 

Implementation  of  this  research  prototype  is  not  expected  to  meet  difficulties  in  implementation  as 
MEDCOM  Reg  25-1  has  been  distributed  in  final  draft  form  excluding  research  projects  from  the 
complexities  of  many  formal  IM  acquisition  process.  Should  this  research  prototype  be  subjected  to 
the  DITSCAP  process  it  is  unlikely  that  the  project  could  successfully  deploy  locally  within  the 
project  timeframe. 

The  Interim  Brigades,  one  of  the  key  target  populations  for  this  project,  continue  to  undergo 
revisions  of  policy,  hardware,  and  equipment.  They  had  been  in  a  relatively  inactive  status  awaiting 
delivery  of  their  primary  vehicle.  However,  this  status  has  changed  and  they  are  now  attempting  to 
make  up  for  previous  downtime  making  their  active  participation  less  viable.  The  second  Interim 
Brigade,  however,  is  now  "standing-up"  and  may  better  meet  project  requirements.  Associated  with 
the  brigades,  the  MC4  Project  Team,  has  deployed  initial  hardware.  They  are  resistant  to  utilizing 
their  hardware  for  other  software  and  providers  are  resistant  to  having  multiple  similar  hardware 
devices  they  have  to  be  responsible  for.  We  are  attempting  to  solve  this  through  the  efforts  to  have 
the  electronic  record  become  an  official  enterprise  system  which  will  have  formal  approved 
requirements  for  MC4  and  TMIP  integration. 

Finally,  the  absence  of  an  approved  AMEDD  wireless  standard  has  impacted  upon  the  ability  to 
design  and  fully  consider  implementation  with  a  wireless  device. 


PFs  Problem  Area  Evaluation:  : 
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LIFE-CYCLE 


The  second  half  of  the  project  will  continue  with  the  prototyping  of  the  electronic  record.  We  will  be 
implementing  the  FDR-BH  in  the  Behavioral  Health  Clinic  at  Madigan  Army  Medical  Center  and 
with  the  3BDE  Mental  Health  Officer.  We  will  utilize  the  PIC  to  avoid  system  interconnectivity 
issues.  Once  in  place,  clinic  staff  will  be  trained  to  use  the  software  and  the  research  protocol  will 
begin.  After  the  software  is  successfully  installed  for  local  functioning,  we  will  focus  on  implementing 
the  program  on  the  Internet.  Simultaneous  to  this  research  effort  we  are  moving  to  integrate  the 
features  and  functionality  into  an  enterprise  system.  Army  Behavioral  Health  is  establishing  an  IPX 
to  develop  and  field  an  enterprise  solution.  This  effort  will  work  to  modify  an  existing  ORD  and 
integrate  fielding  with  a  USAMISSA  program  to  optimize  development  cycle  and  field  the  initial 
product,  as  this  project  continues  to  prototype  and  support  newly  evolving  requirements.  This  IPT 
will  work  in  coordination  with  the  Health  Policy  and  Services  Research  Directorate  to  integrate  with 
the  appropriate  general  officer  integration  committees. 

Efforts  will  continue  to  integrate  the  products  with  TMIP  and  MC4  and  push  requirements 
definition  to  CHCS-II  for  implementation  within  their  Block  3  development  cycle.  The  project  PI 
has  the  Army  lead  for  CHCS-II  behavioral  health  requirements  definition  and  validation. 

Additionally,  we  have  initiated  several  new,  related  efforts.  We  have  initiated  development  of  a 
“Disaster  Response  Toolkit”  to  support  providers  responding  to  a  range  of  situations.  This  is  being 
worked  on  as  a  collaborative  project  with  the  VA  National  Centers  for  PTSD.  The  project  is 
continuing  to  explore  wireless  deployment  to  support  field  access  on  deployment.  While 
controversial,  due  to  standards  approval  decisions  and  security  challenges,  this  project  continues  to 
push  forward  in  this  area.  We  have  initiated  the  dialogue  and  processes  necessary  to  integrate  with 
the  Army’s  MC4  hardware  both  with  the  local  Ft  Lewis  MC4  project  personnel  and  with  the  MC4 
PM  and  TMIP  PM.  We  continue  with  efforts  to  integrate  cognitive  testing  into  the  overall  system 
and  to  establish  a  mechanism  within  this  program  to  support  ongoing  behavioral  health  monitoring 
and  surveillance  through  deployment  surveillance  measures  and  development  toward  a  behavioral 
health  preventive  maintenance  checks  and  services  model  (BH-PMCS). 

We  have  focused  our  research  protocol  on  user  acceptance  and  formal  user  evaluation  of  the 
electronic  record.  This  protocol  has  MAMC  IRB  approval  and  is  awaiting  final  MRMC  approval. 


Pi  s  Life-Cycle  Evaluation:  : 
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DELIVERABLES 


Evolving  external  challenges  have  resulted  in  a  reprioritization  of  project  deliverables  and  a  focus  on 
support  services  that  can  be  provided  without  Internet  connectivity.  This  has  not  been  to  the 
exclusion  of  Internet  based  health  care  services,  rather  as  an  additional  important  focus.  Thus,  at 
this  point,  the  project  is  a  two-pronged  effort.  One  major  thrust  remains  development  of  Internet 
behavioral  health  services.  The  other  major  effort  is  focused  on  meeting  behavioral  health  needs 
off-line.  These  are  mutually  supporting  foci  but  have  added  to  project  complexity. 

Development  of  an  integrated,  Internet  enabled  outcome  measure  for  behavioral  health  -  This 
measure,  the  Symptom  Checklist  and  Outcome  Rating  Scale  (SCORS),  has  been  piloted  and  initial 
normative  data  collection  and  cross-validation  studies  are  underway.  The  interest  in  this  government 
owned  measure  has  been  such  that  additional  cross  validation  studies  will  be  conducted  during  the 
second  half  of  this  project  with  Eisenhower  Army  Medical  Center,  Walter  Reed  Army  Medical 
Center,  and  Tripler  Army  Medical  Center.  This  measure  will  also  be  integrated  into  the  enterprise 
record  system  elaborated  on  below. 

Development  of  a  deployable  mental  health  computer-based  patient  record  -  The  project  team 
identified  this  as  a  key  component  of  the  project.  The  team  documented  provider  needs,  analyzed 
JACHO  requirements,  and  developed  initial  functional  requirements.  A  prototype  complete 
computer-based  mental  health  record  has  been  developed,  with  the  assistance  of  TATRC,  building 
on  the  Field  Deployable  Record  (FDR).  This  record  system  prototype  is  undergoing  programming  in 
coordination  with  TATRC  and  is  being  fielded  with  rapid  development  prototype  support. 

The  FDR  was  used  as  the  development  platform  for  the  computer-based  mental  health  record  for 
several  reasons.  First,  the  FDR  is  built  on  the  Java  platform  and  utilizes  XML  technologies,  which 
allows  for  platform  independent  and  Internet-optimized  integration.  Second,  the  FDR  integrates 
with  the  Personal  Information  Carrier  (PIC).  The  PIC  supports  continuity  of  care  across  behavioral 
health  organizations  and  between  garrison  and  deployment.  The  PIC  also  obviates  the  needs  for 
Internet  connectivity  and  the  project  development  challenges  associated  with  such.  This  has  been 
particularly  important  with  the  evolving  Internet  security  threats. 

This  electronic  record  system  prototype  has  successfully  proceeded  far  beyond  what  would  be 
reasonably  expected  within  this  project  timeframe.  While  still  in  its  alpha  version  it  has  been 
deployed  to  Operation  Bright  Star  to  evaluate  its  ability  to  support  behavioral  health  on  a 
deployment.  It  has  been  tasked  to  support  the  behavioral  health  response  to  Operation  Noble  Eagle 
and  has  the  subsequent  Operation  Solace.  It  has  been  tasked  to  support  Operation  Enduring 
Freedom  although  the  ultimate  deployment  may  have  been  interrupted  by  J 6  concerns  over  limited 
technology  support  in  theater  and  the  last  minute  nature  of  the  MEDCOM  request.  This  record 
system  has  also  received  the  interest  of  the  Army  Combat  Stress  Control  community,  which  is 
looking  to  ultimately  use  it  to  support  their  needs. 

Based  on  the  clear  need  this  system  meets  we  have  developed,  with  the  assistance  of  TATRC,  an 
Integrated  Product  Team  (IPT)  recommendation  and  Charter  for  development  of  this  as  an 
integrated  AMEDD  behavioral  health  system.  The  AMEDD  behavioral  health  leadership  has 
endorsed  this  concept  and  is  proceeding  with  this  plan.  To  best  leverage  extant  behavioral  health 
investment  we  will  attempt  to  work  with  existing  ORD  to  modify  this  to  meet  our  needs.  We  are  also 
exploring  the  ability  to  integrate  this  system  with  other  Internet  enabled  mental  health  initiatives, 
including  the  TricareOnline/E-Health  Portal  and  AKO.  An  initial  MOA  is  under  development  with 
AKO  to  explore  hosting  and  security  support. 


PFs  Deliverables  Evaluation:  :  Deliverable  is  on  schedule  per  Proposal 
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Element  of  Resource  (EOR) 

Travel  2100 

1ST  Quarter 

Oct  1  -  Dec  31 

$0.00 

2nd  Quarter 
Jan  1  -  Mar  31 
$0.00 

Shipping  2200 

$0.00 

$0.00 

Rent  &  Communications  2200 

$0.00 

$0.00 

Contract  for  Services 

2500 

$93,625.00 

$0.00 

Supplies 

2600 

$0.00 

$0.00 

Equipment 

3100 

$0.00 

$4,000.00 

Financial  Narrative: 


The  project  was  funded  at  a  level  of  $200,000.  This  was  reduced  from  the  requested  $438,000.  Funds 
were  efficiently  transferred  to  MAMC  via  MIPR  and  allocated  to  a  unique  APC  at  MAMC. 
Obligations  and  accounting  have  been  managed  by  the  PI  with  the  assistance  of  Departmental  and 
project  personnel.  Project  rescoping  was  conducted  to  maximize  the  value  of  project  delivery.  The 
current  budget  estimates  are  as  follows: 

Project  Mgmt/Clinical  Offset:  40,000  Project  Assistant/Programming:117,250  Equipment:  21,000 
Software:  9,000  Travel/Implementation:  13,000 

To  date,  funds  spent  include:  Research  Assistant:  53,625  Project  Mgmt/Clinical  Offset:  40,000 
Software:  4,000 

Fund  expenditure  is  generally  in  line  with  the  revised  estimates  as  submitted  in  response  to  the 
budget  decrement.  Project  Assistant/Programming  resources  will  remain  over  two  full  years. 

This  system  has  potential  applicability  across  the  DOD.  One  method  to  accomplish  this  is  to  utilize 
the  TricareOnline.com  system.  This  has  been  proposes  as  an  FY02  project.  That  project  would 
provide  a  better  estimate  of  the  costs  associated  with  a  broad  based  fielding  and  enterprise 
integration. 


Pi's  Financial  Evaluation:  :  Deliverable  is  on  schedule  per  Proposal 


*  END  OF  REPORT  * 
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